The objective optics used in digital cameras are typically designed so as to minimize the optical point spread function (PSF) and maximize the modulation transfer function (MTF), subject to the limitations of size, cost, aperture size, and other factors imposed by the camera manufacturer. The PSF of the resulting optical system may still vary from the ideal due to focal variations and aberrations.
A number of methods are known in the art for compensating for such PSF deviations by digital image processing. For example, U.S. Pat. No. 6,154,574, whose disclosure is incorporated herein by reference, describes a method for digitally focusing an out-of-focus image in an image processing system. A mean step response is obtained by dividing a defocused image into sub-images, and calculating step responses with respect to the edge direction in each sub-image. The mean step response is used in calculating PSF coefficients, which are applied in turn to determine an image restoration transfer function. An in-focus image is obtained by multiplying this function by the out-of-focus image in the frequency domain.
U.S. Pat. No. 7,077,810 to Alon et al., which is assigned to the assignee of the present application and is incorporated herein by reference, describes a method and system for processing a distorted digital image B that is a convolution of an undistorted image F and a point spread function. Noise is removed from the image B so as to produce an image B′ of reduced noise. The image F is then obtained based upon a calculation involving the image B′.
US Patent Application Publication 2007/0236573 to Alon et al., which is incorporated herein by reference, describes a method for designing a camera, which includes objective optics for forming an image on an electronic image sensor and a digital filter for filtering an output of the image sensor. The method includes estimating an enhancement of the image that can be accomplished using the digital filter. A target optical specification for the camera is processed responsively to the estimated enhancement so as to determine a modified optical specification, and the objective optics are designed responsively to the modified optical specification.
PCT International Publication WO 2004/063989, whose disclosure is incorporated herein by reference, describes an electronic imaging camera, comprising an image sensing array and an image processor, which applies a deblurring function—typically in the form of a deconvolution filter (DCF)—to the signal output by the array in order to generate an output image with reduced blur. This blur reduction makes it possible to design and use camera optics with a poor inherent PSF, while restoring the electronic image generated by the sensing array to give an acceptable output image.
Low-cost color video cameras typically use a single solid-state image sensor with a multi-colored mosaic filter overlay. A mosaic filter is a mask of miniature color filter elements in which a filter element is positioned in front of each detector element of the image sensor. The filter elements in the mosaic filter generally alternate between the primary RGB colors, or between the complementary colors cyan, magenta and yellow. One common type of color mosaic filter is called a “Bayer sensor” or “Bayer mosaic,” which is described in U.S. Pat. No. 3,971,065, whose disclosure is incorporated herein by reference.
Processing the image produced by a mosaic image sensor typically involves reconstructing the full color image by extracting three color signals (red, green and blue) from the sensor output. An image signal processor (ISP) processes the image sensor output in order to compute luminance (Y) and chrominance (C) values for each pixel of the output image. The ISP then outputs these values (or the corresponding R, G and B color values) in a standard video format.
PCT International Publication WO 2007/054931, which is assigned to the assignee of the present patent application and whose disclosure is incorporated herein by reference, describes methods and devices for image enhancement in the mosaic domain. A mosaic image sensor outputs a stream of pixel values belonging to a plurality of input sub-images, each of which is due to light of a different, respective color that is incident on the sensor. An image restoration circuit filters the pixel values in each of the input sub-images so as to generate corresponding output sub-images with enhanced quality, such as with reduced blur. An image signal processor (ISP) then combines the output sub-images so as to generate a color video output image.